Fix panic when recording apps that output non-UTF-8 bytes#259
Fix panic when recording apps that output non-UTF-8 bytes#259ibigbug merged 1 commit intoWatfaq:mainfrom
Conversation
- Use WriteConsoleW instead of std::io::stdout() to bypass UTF-8 validation - Buffer incomplete UTF-8 sequences across chunk boundaries to prevent replacement characters when multi-byte chars are split Fixes Watfaq#243
SteveSandersonMS
force-pushed
the
fix-non-utf8-stdout-panic
branch
from
568d531 to
ef2a6df
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes a panic that occurs when recording applications that output non-UTF-8 byte sequences on Windows. The issue stems from Windows console in UTF-8 mode rejecting invalid UTF-8 sequences through Rust's standard output validation.
Changes:
- Replaces
std::io::stdout().write()with directWriteConsoleWcalls to bypass Rust's UTF-8 validation - Implements buffering logic to handle incomplete UTF-8 sequences that may be split across chunk boundaries
- Ensures proper Unicode handling by converting valid UTF-8 strings to UTF-16 before writing to console
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #[cfg(windows)] | ||
| let stdout_handle: HANDLE = | ||
| unsafe { GetStdHandle(STD_OUTPUT_HANDLE).expect("failed to get stdout handle") }; | ||
|
|
||
| // Buffer for incomplete UTF-8 sequences split across chunk boundaries | ||
| let mut pending_bytes: Vec<u8> = Vec::new(); | ||
|
|
||
| loop { |
There was a problem hiding this comment.
The stdout handle is retrieved once outside the loop, but if the handle becomes invalid during execution (e.g., console detachment), subsequent writes will fail. Consider retrieving the handle inside the loop or adding error handling to detect and recover from invalid handle states.
| #[cfg(windows)] | |
| let stdout_handle: HANDLE = | |
| unsafe { GetStdHandle(STD_OUTPUT_HANDLE).expect("failed to get stdout handle") }; | |
| // Buffer for incomplete UTF-8 sequences split across chunk boundaries | |
| let mut pending_bytes: Vec<u8> = Vec::new(); | |
| loop { | |
| // Buffer for incomplete UTF-8 sequences split across chunk boundaries | |
| let mut pending_bytes: Vec<u8> = Vec::new(); | |
| loop { | |
| #[cfg(windows)] | |
| let stdout_handle: HANDLE = | |
| unsafe { GetStdHandle(STD_OUTPUT_HANDLE).expect("failed to get stdout handle") }; |
There was a problem hiding this comment.
@SteveSandersonMS do you think this comment makes sense? (i'm not a Windows console expert :))
ibigbug
left a comment
ibigbug
left a comment
There was a problem hiding this comment.
wow thank you so much! bar the Copilot comment
2 participants
Windows console in UTF-8 mode rejects non-UTF-8 byte sequences via
std::io::stdout().write(), causing a panic.This PR:
WriteConsoleWto write directly to the console handle, bypassing Rust's UTF-8 validationFixes #243